Execution profiling blueprints

M O Re ct an gl eS ha pe > > co m pu te Ex te nt Ha vi ng Ch ild re nF or : Figure 12. Behavioral Evolution Blueprint over Versions 200 and 352 rooted in MORoot>>applyLayout. The optimization described in the previous sections are part of a major effort to optimize Mondrian. This effort started after Version 200. To assess our optimizations on the long term, we use Version 200 as the referential. When compared with Version 352, we identify an optimization Copyright c © 2010 John Wiley & Sons, Ltd. Softw. Pract. Exper. (2010) Prepared using speauth.cls DOI: 10.1002/spe EXECUTION PROFILING BLUEPRINTS 19 implemented somewhere between Version 200 and 352: method MORoot>>applyLayout is faster in Version 352 and its source code has slightly changed (some methods have been renamed), changing from the version on the left hand side to the one on the right hand side (bold font indicates differences): self shapeBounds self shapeBoundsAt: self shape at: self shape put: (0@0 corner: 0@0). put: (0@0 corner: 0@0). self do: [ :each | self do: [ :each | each applyLayout ]. each applyLayout ]. self layout applyOn: self. self layout applyOn: self. self do: [:each | self nodesDo: [:each | self bounds corner: self bounds corner: (self bounds corner max: (self bounds corner max: (each extent + each origin)). (each extent + each origin)). ]. ]. With the Behavioral Evolution Blueprint in Figure 12 only three methods remain candidates to the source of the improvement. First, MOAbstractLayout >>initializeConnectionPositions, but its execution time is too small to cause the global improvement. Second, MORectangleShape >>computingExtentHavingChildrenFor and third MONode>>translateTo. The third method decreases from 64ms to 1ms. Its source code was changed to include memoization, hence calling less times MOGraphElement>>bounds—a method that later added caching as seen in Section 3.2. Comparing the profiling obtained with Version 353 against the one obtained with Version 200 retrospectively assesses the performance of Mondrian after 7 months (Version 200 was released on May 14, 2009 and Version 352 on January 25, 2010). Navigating through history. The Behavioral Evolution Blueprint easily compares two execution profiles, each profile supposedly obtained from a particular software version. The profile that is compared with the anterior profile may be chosen by means of a contextual menu. This menu allows for moving through the history in a frame-by-frame basis. Clicking the “next” comparison button (not shown in the figures) for a particular method renders a new blueprint where the second profile of the previous blueprint becomes the first, and is compared to the profile associated with the next version available. A “previous” button is also available to navigate in the opposite direction. Note that the analysis data for the previous versions are stored and reused; navigating through the history of the profile is therefore without noticeable lag. In order to navigate the profiling history, we require a set of profiles taken over the execution of a particular code snippet throughout several versions of a software system. After selecting a particular set of versions where profiles are to be taken, we sort them from the oldest to the newest. The set of profiles begins with the first version’s profile. Steps (ii) to (iv) are then repeated for each subsequent version of the system, loading, profiling, and comparing it with the last available. The profiling information is then stored in a similar approach to the one proposed in [11] for source code history. Experience on Mondrian. We applied the previous algorithm to obtain a behavioral evolution blueprint analyzing the method MORoot >>applyLayout through a set of 22 versions, ranging from 400 to 570. Navigation through the set of blueprints obtained made visible variations in the execution time associated with the cache implementation. Between Versions 450 and 460, the call of a method MONode >>resetCacheInEdges in method MONode >>translateBy:bounded: was introduced as an effort to homogenize the various cache mechanisms Mondrian provides. This new method produced an increase of the execution time for MORoot >>applyLayout. The time increase of MORoot >>applyLayout is, in contrast, reduced in Version 589 compared to Version 510. This is caused by changes in the method MOGraphElement >>applyLayout. Version Copyright c © 2010 John Wiley & Sons, Ltd. Softw. Pract. Exper. (2010) Prepared using speauth.cls DOI: 10.1002/spe 20 A. BERGEL, F. BAÑADOS, R. ROBBES, W. BINDER FSGitFilesystem>>commit: FSGitFilesystem>>basicCommit:to: Figure 13. Behavioral evolution blueprint of Version 80 compared with Version 70 of GitFS. 589 of this method contains an extra initialization of the shape, which significantly reduces the cost of layouting nodes: MOGraphElement >>applyLayout | b | self resetMetricCachesResursively. b := self hasChildren ifTrue: [ 0 @ 0 extent: 0 @ 0 ] ifFalse: [ self shape computeBoundsFor: self ]. self shapeBoundsAt: self shape put: b. ”Added in Version 589” self hasChildren ifTrue: [ self do: [ :each | each applyLayout ]. self layout applyOn: self ]. Complementary case study. Beside Mondrian, we have used the behavioral evolution blueprint on a number of different Pharo projects. For example, GitFS∗ is an implementation of Git in Pharo. At the time this article is being written, 91 different versions of GitFS are published. Git is a distributed revision control system with an emphasis on speed†. As a representative benchmark for a revision control system, we measured a sequence of 500 commits made on a file present in memory (no system primitives are therefore involved). The method FSGitFilesystem>>commit: is responsible to commit a change into a new file. This method is the entry point to realize a Git commit operation. For Version 70, our benchmark made the #commit: method takes 2 541 milliseconds to execute. For Version 80, #commit: took 4 255 milliseconds, which represents a slowdown of 67%‡. Figure 13 visually represents the cause of the slowdown. The source code of #commit: has not changed from Version 70 to Version 80. However, #commit: invokes #basicCommit:to: and the ∗http://www.squeaksource.com/GitFS.html †http://git-scm.com ‡= (4255− 2541) ∗ 100/2541 Copyright c © 2010 John Wiley & Sons, Ltd. Softw. Pract. Exper. (2010) Prepared using speauth.cls DOI: 10.1002/spe EXECUTION PROFILING BLUEPRINTS 21 source code of this method has changed. It went from: ”Version 70” ”Version 80” basicCommit: aMessage to: branchName basicCommit: aMessage to: branchName | commit parents | | commit parents | self assert: modManager hasModifications. modManager hasModifications ifFalse: [ ... ]. modManager processBlobs. modManager processBlobs. ... ... revision ifNotNil: [ parents add: revision ]. revision ifNotNil: [ parents add: (repository objectWithSignature: revision signature) ] revision := commit. revision := repository objectWithSignature: commit signature. Of the two modifications, the addition of the signature when committing is responsible of the slowdown. To confirm our findings, we contacted the author of GitFS to determine whether he agreed on our findings; he answered positively. Summary. We propose the Behavioral evolution blueprint as an effective and intuitive visualization that complements Structural distribution blueprint and Behavioral distribution blueprint, two visualizations of a unique profile snapshot. This third blueprint focuses on differencing profiles of two or more snapshots. A visualization always differentiates two profiles of the same application and benchmark, but for different software versions. Behavioral evolution blueprint helps tracking down the cause of a performance increase or decrease. The effectiveness of behavioral evolution blueprint depends on the structural difference between the profiled version. Versions need to be “close enough” to give a useful meaning to the comparison. This point is discussed further in the following section (Section 5).